[orion-server]:Add a buffer queue for task dispatch @Ivanbeethoven#1325
Conversation
Signed-off-by: Xiaoyang Han <lux1an@qq.com>
Signed-off-by: Han Xiaoyang <lux1an@qq.com>
Signed-off-by: Xiaoyang Han <lux1an@qq.com>
Signed-off-by: Xiaoyang Han <lux1an@qq.com>
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
Signed-off-by: Lux1an <lux1an@qq.com>
There was a problem hiding this comment.
Pull Request Overview
This PR introduces a comprehensive task scheduler with a buffer queue for managing build tasks in the orion-server. The implementation moves from direct worker assignment to a queue-based system that can handle task queuing when workers are busy, automatic task dispatching, and improved worker management.
Key changes:
- Replaces direct worker assignment with a sophisticated TaskScheduler that includes a FIFO task queue
- Adds queue management with configurable limits, timeout handling, and automatic cleanup of expired tasks
- Implements event-driven task processing with background queue manager for efficient resource utilization
Reviewed Changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| orion-server/src/scheduler.rs | New comprehensive scheduler module with TaskQueue, TaskScheduler, and related data structures |
| orion-server/src/api.rs | Refactored to use TaskScheduler, added queue statistics endpoint, and integrated queue-aware task handling |
| orion-server/src/server.rs | Updated to use new AppState structure and start queue manager background task |
| orion-server/src/main.rs | Added scheduler module declaration |
| orion-server/SCHEDULER_README.md | Comprehensive documentation for the new scheduler system |
| orion-server/Cargo.toml | Added thiserror dependency for error handling |
Comments suppressed due to low confidence (2)
orion-server/src/api.rs:412
- Same issue as in scheduler.rs -
random_rangemethod may not exist. Userng.gen_range(0..idle_workers.len())instead.
rng.random_range(0..idle_workers.len())
orion-server/src/api.rs:411
- Same issue as in scheduler.rs - use
rand::thread_rng()instead ofrand::rng().
let mut rng = rand::rng();
| let mut rng = rand::rng(); | ||
| rng.random_range(0..idle_workers.len()) | ||
| }; | ||
| let chosen_id = idle_workers[chosen_index].clone(); |
There was a problem hiding this comment.
The random worker selection logic is duplicated between this function and handle_immediate_task_dispatch. Consider extracting this into a shared helper function to reduce code duplication.
| let chosen_id = idle_workers[chosen_index].clone(); | |
| // Randomly select an idle worker using the helper function | |
| let chosen_id = match select_random_worker(&idle_workers) { | |
| Some(id) => id, | |
| None => return Err("No idle workers available".to_string()), | |
| }; |
| // Randomly select an idle worker | ||
| let chosen_index = { | ||
| let mut rng = rand::rng(); | ||
| rng.random_range(0..idle_workers.len()) |
There was a problem hiding this comment.
The random_range method may not exist in the current rand crate API. Consider using rng.gen_range(0..idle_workers.len()) instead, which is the standard method for generating random numbers in a range.
| rng.random_range(0..idle_workers.len()) | |
| let mut rng = rand::thread_rng(); | |
| rng.gen_range(0..idle_workers.len()) |
| // Randomly select an idle worker | ||
| let chosen_index = { | ||
| let mut rng = rand::rng(); | ||
| rng.random_range(0..idle_workers.len()) |
There was a problem hiding this comment.
The rand::rng() function may not exist in the current rand crate API. Consider using rand::thread_rng() instead for thread-local random number generation.
| rng.random_range(0..idle_workers.len()) | |
| let mut rng = rand::thread_rng(); | |
| rng.gen_range(0..idle_workers.len()) |
| pub fn get_build_log_dir() -> &'static str { | ||
| use once_cell::sync::Lazy; | ||
| static BUILD_LOG_DIR: Lazy<String> = | ||
| Lazy::new(|| std::env::var("BUILD_LOG_DIR").expect("BUILD_LOG_DIR must be set")); |
There was a problem hiding this comment.
Using once_cell::sync::Lazy for environment variable access can panic if BUILD_LOG_DIR is not set. Consider using a Result type or providing a default value to handle missing environment variables gracefully.
| Lazy::new(|| std::env::var("BUILD_LOG_DIR").expect("BUILD_LOG_DIR must be set")); | |
| /// Get build log directory. Uses the BUILD_LOG_DIR environment variable if set, otherwise defaults to "./build_logs". | |
| pub fn get_build_log_dir() -> &'static str { | |
| use once_cell::sync::Lazy; | |
| static BUILD_LOG_DIR: Lazy<String> = | |
| Lazy::new(|| std::env::var("BUILD_LOG_DIR").unwrap_or_else(|_| "./build_logs".to_string())); |
No description provided.